Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add list-outputs #248

Merged
merged 21 commits into from
Apr 14, 2023
Merged

Add list-outputs #248

merged 21 commits into from
Apr 14, 2023

Conversation

bshifaw
Copy link
Collaborator

@bshifaw bshifaw commented Mar 16, 2023

Add list-outputs command
Add unit and integrations tests

@bshifaw bshifaw added the Cromshell 2 Issues related to Cromshell 2.0 label Mar 16, 2023
@bshifaw bshifaw self-assigned this Mar 16, 2023
@bshifaw bshifaw marked this pull request as ready for review March 24, 2023 15:00


class TestListOutputs:
@pytest.mark.parametrize(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the pytest parameters below, I'm using the same workflow to test different options of list-output. This runs the same workflow multiple times (redundant). I'm thinking of maybe running the workflow once, saving the workflow id in a global variable then have a different function that would run through the different list-output command

Copy link
Contributor

@SHuang-Broad SHuang-Broad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's also make sure #249 is reflected here.

@@ -9,10 +9,11 @@
from cromshell.utilities import cromshellconfig


def run_cromshell_command(command: list, exit_code: int):
def run_cromshell_command(command: list, exit_code: int, options: list = None):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's document what these parameters are.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done, also changed name to subcommand_options

command_with_options = command.copy()
if options:
for item in options:
command_with_options.insert(1, item)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what if the options are meant to be in order? This seems to reverse that.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updated, to take this into account

class TestListOutputs:
"""Test the execution list-outputs command functions"""

# def get_workflow_level_outputs(config) -> dict:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a self-reminder? If so, let's be a bit explicit in the comments.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oops, that should have been deleted

],
],
)
def test_filer_outputs_from_workflow_metadata(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo? filter?

src/cromshell/list_outputs/command.py Show resolved Hide resolved
return filer_outputs_from_workflow_metadata(workflow_metadata)


def filer_outputs_from_workflow_metadata(workflow_metadata: dict) -> dict:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you mean "filTer"?

print(f"{i}{output_name}: {output_value}")


def is_path_or_url_like(in_string: str) -> bool:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm thinking about future-proofing us a little here: how about we classify the in_string?
I mean classify it as a local regular path, GCS path, or Azure path, or not a path at all.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Classification would be useful in a command like fetch-outputs where we need to know the classification before downloading the output. Since we're not downloading anything in this function, we'd only be interested in knowing whether its a file.

return requests_out.json()
else:
http_utils.check_http_request_status_code(
short_error_message="Failed to retrieve workflow outputs.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It'd be better if we also include the ID of this workflow that fails.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

src/cromshell/list_outputs/command.py Show resolved Hide resolved
src/cromshell/list_outputs/command.py Show resolved Hide resolved
@bshifaw bshifaw merged commit abd12ba into cromshell_2.0 Apr 14, 2023
@bshifaw bshifaw deleted the bs_add_list_outputs branch April 14, 2023 14:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Cromshell 2 Issues related to Cromshell 2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Make list-outputs give back not only files, but naive types too
2 participants